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PREFACE 


The  human  factors  involved  in  the  operation  and  maintenance 
of  guided  missiles  are  variables  which  must  be  considered  in 
determining  the  reliability  of  the  weapon.  The  over-all  relia- 
bility is  a resultant  of  the  reliabilities  of  the  individual 
components  of  the  weapon  system.  The  reliabilities  of  the  c oppo- 
nent s of  the  system  are,  in  turn,  a function  of  the  following 
variables: 


1.  The  basic  principles  of  the  operation  of  the  weapon. 

2.  The  design  of  the  weapon. 

3.  Engineering  testing  and  evaluation. 

4.  Fabrication  and  production. 

5.  Production  inspection  and  quality  control . 

6.  Transportation  (handling),  assembly  and  stowage. 

7*  Preflight  check-out. 

8.  Trouble-shooting. 

9.  Servicing  and  repair. 

10.  The  tactical  use  of  the  weapon. 

Variables  1 through  5 are  usually  considered  to  be  the  respon- 
sibility of  the  missile  designer  manufacturer  and  Navy  engi- 
neering, human  engineering,  and  inspection  agencies.  It  is,  how- 
ever, variables  6 through  9 in  the  above  list  which  largely  re- 
flects the  adequacy  of  personnel  selection,  training,  and  Job  struc- 
ture. The  proficiency  of  Naval  personnel  in  performing  the  tasks 
associated  with  vai  tables  6 through  9 significantly  contribute  to 
the  reliability  of  a particular  guided  missile. 
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and  policy  on  sound  objective  data,  the  Personnel  Analysis  Division 
of  the  Bureau  of  Naval  Personnel  has  nlanned  a urocraia  of  systematic 


guided  missile  personnel  research.  The  procedures  developed  and 
the  data  obtained  In  this  program  can  contribute  to  the  following: 


1.  The  specification  of  the  knowledges  and  skills  required 
by  personnel  to  perform  the  Job  behaviors  associated 
with  the  tasks  required  by  the  new  weapon. 

2.  A comparison  of  the  knowledges  and  skills  required  by 
the  new  weapon  with  the  knowledges  and  skills  available 
in  existing  manpower. 
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3*  Roconmendations  for  an  efficient  job  structure  for 
personnel  concerned  with  the  new  weapon. 

4.  The  development  of  on-the-job  criteria  of  job  effec- 
tiveness. 

5*  Reccnaendations  for  the  selection  and  training  of 
personnel  to  operate  and  maintain  the  new  weapon. 

6'  The  development  and  construction  of  tests  of  job 
proficiency. 

7«  RecGsmendations  for  the  most  efficient  organization 
and  use  of  operations  and  maintensnce  manpower. 

In  the  guided  missile  personnel  research  program  during  the 
period  from  July  1952  to  July  1953*  the  project  undertaken  by  the 
American  Institute  for  Research  has  been  concerned  primarily  with 
the  following  objectives: 

1.  The  compilation  of  task  information  for  Terrier  missile 
activities. 

2.  The  developn*en+  of  testing  procedures  and  methodology 
for  measuring  the  proficiency  of  personnel  who  perform 
duties  associated  with  the  operation  and  maintenance 
of  Terrier. 

The  work  accomplished  toward  these  objectives  are  reported  in 
the  following  two  reports: 

A.  Guided  Missile  Personnel  Research;  Report  No.  2 

Part  One:  Collecting  and  Compiling  Task  Information 

for  Newly  Developed  Guided  Missiles. 

Part  Two:  A Cangpilatica  of  Task  Information  for 

Terrier  Missile  Activities. 

B.  Guided  Micelle  Personnel  Research:  Report  No.  3 

A Proficiency  tfest  Battery  for  Guided  Missile 

Technicians. 

Report  No.  3 is  presented  here. 
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ABSTRACT 


This  re port  presents  and  discusses  the  development  of  a series 
of  proficiency  tests  for  Terrier  guided  missile  technicians.  These 
tests  can  he  used  to  assess  men  at  the  end  of  training  or  may  be 
used  as  training  adds  to  facilitate  learning. 

Three  kinds  of  multiple  choice  tests  were  developed: 

1.  A Testing  anu  Adjustment  Test  which  measures  proficiency 
in  the  performance  of  specific  testing,  adjustment  and 
servicing  procedure  sj  in  the  interpretation  of  test  and 
adjustment  results  j and  in  the  making  of  decisions  con- 
cerning the  proper  equipment  or  procedures  to  enploy. 

2.  A Trouble-Shooting  Test  which  measures  proficiency  in 
the  interpretation  of  systems  of  malfunctioning  and  in 
isolating  trouble  to  particular  components  of  a system. 

3.  A Knowledge  of  Operation  Test  which  measure  proficiency 
in  the  application  and  adaptive  use  of  the  principles 
of  equipment  functioning. 

A new  technique,  called  the  Trouble-Shooting  Formtoard,  is 
presented  which  has  wide  applicability  as  both  a proficiency  meas- 
urement and  a training  aid  device.  The  Trouble-Shooting  Form- 
board  realistically  permits  a trainee  to  observe  e symptom  of 
equipment  malfunctioning,  to  perform  a series  of  checks  and  ob- 
tain information  from  these  checks,  and  on  the  basis  of  this  in- 
formation to  investigate  units  until  the  defective  unit  is  lo- 
cated. This  is  accomplished  without  the  use  of  the  actual  equip- 
ment. 

Hie  multiple-choice  tests  and  the  formboard  items  ere  pre- 
sented in  appendices  included  in  this  report. 
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imOEf'CTIOh' 


The  ultimate  objective  of  a personnel  program  associated  with 
a particular  guided  missile  is  to  maximize  the  reliability  of  this 
weapon.  Reliability  is  enhanced  when  all  variables  influencing  the 
weapon  are  controlled  within  the  necessary  tolerances;  a list  of 
these  variables  is  given  in  the  preface  to  this  report.  The  out- 
put of  a training  program,  i.e.,  the  quality  of  performance  of  the 
personnel  trained  in  the  program,  directly  affects  weapon  relia- 
bility. An  effective  way  of  controlling  this  source  of  variation 
is  through  the  use  of  Job-oriented  proficiency  examinations.  Pro- 
ficiency tests  provide  a standard  to  which  the  finished  product, 
the  trained  technician,  must  conform  before  being  sent  to  perform 
his  function  in  contributing  to  weapon  reliability.  It  is  impor- 
tant that  control  of  the  quality  of  the  technician  should  take 
place  on  two  levels:  (l)  a content  level  which  is  a function  of 

the  subject  matter  content  of  the  proficiency  examinations,  and, 

(g ) a behavioral  level  which  is  a function  of  the  kind  of  behaviors 
and  performances  that  are  required  with  respect  to  this  content. 

This  report  discusses  the  development  of  an  experimental  bat- 
tery of  proficiency  tests  for  Terrier  guided  missile  technicians. 
Work  on  this  battery  has  concentrated  on  the  development  of  tests 
that  do  not  require  the  use  of  actual  equipment.  (At  the  present 
stage  of  the  missile  program,  equipment  performance  tests  requiring 
the  use  of  the  critical  gear  which  must  be  diverted  from  weapon 
evaluation  activities  is  not  feasible.)  The  underlying  aim  of  the 
experimental  battery  is  to  develop  tests  which  do  not  require  ac- 
tual equipment  but  which  are  highly  correlated  with  performance 
tasks,  and  to  identify  those  tasks  which  are  and  those  tasks  which 
are  not  practically  amenable  to  testing  without  actual  equipment. 

If  acceptable  validities  are  obtained  for  these  kinds  of  tests, 
it  is  advisable  to  keep  equipment  tests  in  a testing  program  to 
a minimum; 


The  experimental  battery  developed  consists  of  a set  of  pro- 
ficiency tests  which  can  be  used  to  assess  men  at  the  end  of  train- 
ing. However,  in  many  instances  the  bests  may  be  used  as  train- 
ing examinations  to  facilitate  learning.  The  initial  stage  of  be- 
coming proficient,  which  involves  knowledge  of  nomenclature  and 
the  identification  of  parts,  is  not  measured  per  se  in  the  experi- 
mental test  battery;  that  is,  there  is  no  separate  nomenclature 
and  identification  test.  These  behaviors  are  usually  part  of  such 
Job  behaviors  as  checking  and  trouble-shooting,  and  are  tested  as 
such  in  the  test  battery. 
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The  test  battery  concentrates  on  the  technical  jobs  which 
the  guided  missile  technician  performs  cm  the  missile  and  its 
associated  test  and  servicing  equipment.  Guided  missile  Jobs 
which  are  included  in  the  tasks  performed  by  Gunner's  Mates  and 
Fire  Control  Technician*  are  not  covered  in  the  battery*  A de- 
scription of  the  tasks  performed  by  guided  missile  technicians 
in  comparison  with  the  guided  missile  task6  performed  by  these 
other  ratings  is  presented  in  Guided  Missile  Personnel  Research: 
Report  Ro.  2,  Part  Two*  A Caqpllatlon  of  flask  Information  for 
Terrier  Missile  Activities, 


The  present  report  describes  a mul tipis  -choice  iestbattery  con- 
sisting or  three  kinds  of  teste,  namely,  testing  and  alignment, 
trouble-shooting,  and  knowledge  of  operation.  A new  technique 
called  the  trouble-shooting  formboara  is  also  presented.  This 
technique  can  be  employed  both  as  a proficiency  test  and  a train- 
ing device  for  trouble-shooting  tasks. 
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CHAPTER  I 

THE  KCLTIP1JS -CHOICE  TEST  BATTER! 


The  development  of  the  multiple  -choice  portion  of  the  Terrier 
Proficiency  Test  battery  ie  described  iu  three  sectional 

1.  The  Behavior  Measured 

2.  The  Content  Covered 

3.  Test  Construction 


The  Behavior  Measured 


Analysis  of  the  job  of  the  Terrier  missile  technician  indi- 
cates three  major  kinds  of  job  behavior  that  should  be  included  in 
a proficiency  test  battery* 

a.  Testing  and  Adjustment 

b.  Troubleshooting 

c.  Knowledge  of  Operation 

Testing  and.  Adjustment 

Testing  and  adjustment  procedures  involve  job  behavior  such 
as  the  performance  of  specific  manipulatory  procedures,  the  inter- 
pretation of  test  or  adjustment  results,  and  the  making  of 
decisions  concerning  the  proper  equipment  or  procedures  to  employ. 
(Testing  and  adjustment  for  the  Terrier  missile  is  described  in 
Guided  Missile  Personnel  Research  Report  No.  2.  Part  11}  A 
Compilation  of  Task  Information  for  Terrier  Missile  Activities ) 

The  task  of  the  tost  technician  is  to  construct  test  item 
situations  which  evoke  examinee  behavior  that  is  closely  related 
to  the  behavior  required  for  job  performances.  Examples  of  the 
kinds  of  test  item  situations  that  are  employed  in  the  testing  and 
adjustment  test  are  given  below.  For  each  item  type,  the  "situa- 
tion given"  and  the  "examinee  behavior"  on  a multiple-choice  item 
is  described.  The  actual  test  items  are  presented  in  Appendix  A. 
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Situation  Given* 
Examinee  Behavior* 

Situation  Given* 
Examinee  Behavior * 

Situation  Given* 
Examinee  Behaviors 

Situation  Givens 
Examines  Behavior* 

Situation  Given* 
Examinee  Behavior* 

Situation  Given* 
Examinee  Behavior: 

Situation  Given* 

Examinee  Behavior: 

Situation  Given* 
Examinee  Behavior* 

Situation  Given* 

Examinee  Behavior* 

Situation  Given* 
Examinee  Behavior* 

Situation  Given* 

Examinee  Behaviors 

Situation  Given* 
Examinee  Behavior* 

Situation  Given* 
Examinee  Behavior* 

Situation  Given* 
Examinee  Behaviors 


A check  or  adjustment  to  be  made. 

Select  the  test  equipment (s)  used  to  make 
this  check  or  adjustment. 

A component  has  been  installed* 

Select  the  first  step  in  adjusting  it* 

A circuit  condition  and  a desired  change  in 
the  condition* 

Select  the  adjustment  that  can  accomplish 
this. 

An  out-of -tolerance  test  result. 

Select  the  correct  adjustment  procedure. 

A function  to  be  checked  and  the  appropriate 
test  procedure. 

Select  the  correct  test  panel  knob  and 
switch  positions. 

An  adjustment  to  be  made. 

Select  the  correct  test  equipment  dial 
settings. 

A testing  or  servicing  procedure  to  be 
performed. 

Select  the  correct  sequence  of  operations* 

A test  signal  or  other  test  indication. 
Identify  how  the  equipment  la  functioning. 

In  the  course  of  a sequence  of  operations 
a step  is  emitted  or  performed  incorrectly* 
Select  the  consequences  of  this  malpractice. 

Alternate  results  of  a test. 

Select  the  result  that  does  not  indicate  a 
test  equipment  fault. 

A test  set-up  is  presented  which  shows 
component  interconnections* 

Identify  the  connections  between  components* 

A dial  or  knob  adjustment  is  performed. 
Identify  what  this  accomplishes. 

The  name  of  an  equipment. 

Identify  vhat  components  of  this  equipment 
should  be  checked  most  frequently. 

The  name  of  a test  equipment* 

Tdentify  what  it  is  used  to  measure  or  check. 
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Trouble -Shoot ina 


Trouble -« hooting  behavior  involves  the  interpretation  of  in- 
dications or  symptoms  of  malfunctioning;  on  the  basis  of  these 
symptoms,  a technician  isolates  a trouble  to  a particular  compon- 
ent- of  a system*  (Trouble-shooting  behavior  for  the  Terrier 
missile  is  described  in  Guided  Missile  Personnel  Research! 

Report  No.  2,  Part  II.  A Compilation  of  Task  Information  for 
Terrier  Missile  Activities). 

Multiple-choice  test  items  to  measuro  trouble-shooting  pro- 
ficiency present  the  examinee  with  a description  or  indication  of 
malfunctioning  and  require  him  to  select  the  faulty  component* 

F;r  piloted  aircraft  or  eqiipment  that  is  used  over  again,  many 
malfunctions  are  reported  by  the  using  operator;  for  guided 
missiles,  ^It'inctiona  are  observed  as  a result  of  ore -flight 
tests  in  the  course  of  making  adjustments  or  as  a result  of 
assembly  and  launching  difficulties.  Maintenance  data  reports, 
offer  excellent  source  material  for  trouble-,? hobtlng  test  items. 
These  reports  (see  Report  No. 2 ) are  obtained  from  the  field  and 
describe  a casualty,  its  symptom,  how  it  was  located  and  hoy  it 
was  corrected.  The  multiple- choice  trouble-shooting  test  items  in 
the  Terrier  proficiency  tost  battery  are  presented  in  Appendix  A. 

Knowledge  of  Operation 

la  the  military  situation  there  exist  tvo  conflicting  goals 
of  training.  One  goal  is  to  train  a man  quickly  to  operate  and 
maintain  a specific  piece  of  gear;  the  other  goal  is  to  train  a 
man  so  that  he  can  readily  adapt  to  new  equipment,  assimilate 
modifications  in  old  equipment,  and  handle  a piece  of  equipment 
in  emergency  or  non-standard  situations*  The  first  goal  requires 
a specific  kind  of  training;  the  second  goal  a more  general  train- 
ing. A compromise  between  these  two  goals  is  necessary,  and  a 
proficiency  test  battery  should  e»  oriented  toward  of  these 
goals. 


The  Testing  and  Adjustment  Test  and  the  Trouble-Shooting  Test 
are  oriented  toward  specific  on-the-job  tasks.  The  Knowledge  of 
Operation  Test  is  oriented  toward  the  eecond  goal  and  tests  the 
application  of  general  principles  of  equipment  functioning. 
"Application"  is  the  k»y  term  here*  The  examinee  is  not  asked 
primarily  to  recall  principles  he  remembers,  but  to  uss  these 
principles  to  solve  a problem.  The  examinee  is  given  problem 
situations  concerned  with  particular  gear,  in  this  case  Terrier  and 
Terrier  test  equipment,  in  which  he  must  use  principles  underlying 
the  operation  and  functioning  of  this  gear  to  solve  practical 
problems* 
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The  kind  of  knowledge  tested  with  the  Knowledge  of  Operation 
test  can  be  called  tranafarrable  knowledge  and  is  hypothesized  to 
be  characteristic  ox  individuals  who  are  capable  of  adaptive  job 
behavior*  Behavior  of  this  kind  can  be  evoked  in  stimulus  settings 
where  an  individual  cannot  respond  entirely  on  the  basis  of  rote 
learning  or  straight  memory.  In  the  construction  of  items  far 
this  test,  the  attempt  is  made  to  reduce  memory  to  a minimum  by 
giving  the  examinee  all  the  information  he  would  have  on  the  job 
in  terms  of  circuit  diagrams,  pre-test  information,  etc.;  he  is 
then  presented  with  a test  situation  while h requires  the  applica- 
tion of  job-oriented  knowledge.  Test  items  of  low  difficulty 
level  cn  this  kind  of  test  appear  to  be  close  to  straight  memory 
items  and  probably  can  be  answered  either  cn  this  basis  or  through 
a knowledge  of  equipment  operation,  items  of  higher  difficulty 
appear  to  require  the  application  of  knowledge* 

Examples  of  the  kinds  of  test  item  situations  that  are  em- 
ployed in  the  Knowledge  of  Operations  test  are  given  below.  For 
each  item  type,  the  "situation  given"  and  the  " examinee  behavior" 
on  a multiple -choice  item  is  described.  The  actual  test  items 
are  presented  in  Appendix  A. 


Given  Situation: 
Examinee  Behavior: 

Given  Situation: 
Examinee  Behavior: 

Given  Situation: 
Examinee  Behavior  t 

Given  Situation: 
Examinee  Behavior i 

Given  Situation: 
Examinee  Behavior: 

Given  Situation: 

Examinee  Behavior: 
Given  Situation: 

Examines  Behavior: 


Input  to  a circuit  and  a defective  component. 
Identify  the  output  signal. 

An  input  waveform  to  a circuit  or  component. 
Identify  th©  correct  output  waveform. 

A change  in  circuit  input. 

Identify  the  change  in  output. 

A change  in  one  aspect  of  a circuit  that  re- 
sults in  a certain  effect. 

Select  another  change  that  would  produce  the 
same  effect. 

A change  in  circuit  input  from  correct  to 
incorrect* 

Select  the  part  of  the  system  that  would  not 
function  properly. 

A component  becomes  defective  and  operates 
incorrectly* 

Identify  how  the  equipment  now  functions. 

A piece  of  test  gear  and  a circuit  or  compon- 
ent. 

Identify  the  indication  on  the  test  gear  that 
indicates  that  the  component  is  functioning 
p^opsrly. 
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Given  Situation* 
Examinee  Behavior* 

Given  Situation* 

Examinee  Behavior* 

Given  Situation* 

Examinee  Behavior* 

Given  Situation* 
Examinee  Behavior* 

Given  Situation* 
Examinee  Behavior* 

Given  Situation* 
Examinee  Behavior* 


Given  Situation: 
Examinee  Behavior* 

Given  Situation* 
Examinee  Behavior* 

Given  Situations 

Examinee  Behavior* 

Given  Situation* 


Examinee  Behavior* 

Given  Situation* 

Examinee  Behavior* 

Given  Situation* 
Examinee  Behavior* 

Given  Situations 
Examinee  Behavior* 


An  indication  in  a test  instrument. 

Select  the  equipment  condition  that  deter- 
mines this  indication. 

A description  of  equipment  performance  and  a 
change  in  the  circuit. 

Identify  how  the  equipment  performance  changes. 

A description  of  an  equipment  performance. 
Identify  what  occurs  in  a particular  circuit. 

A description  of  an  equipment  performance. 
Identify  what  occurs  in  a particular  circuit 
just  prior  to  this  performance. 

The  name  of  a type  of  circuit. 

Select  the  diagram  that  best  represents  this 
circuit. 

A function  of  a circuit  and  diagrams  of  alter- 
nate circuits. 

Select  the  circuit  that  best  or  correctly 
performs  this  function. 

A block  diagram. 

Identify  the  unlabeled  blocks. 

A block  diagram. 

Identify  connections  between  blocks. 

A description  of  equipment  functioning. 

Select  the  explanation  of  this  functioning. 

A defective  component  and  the  resultant 
effect  upon  equipment  performance. 

Select  the  best  explanation  or  reason  for 
this  result. 

A sequence  of  operations  that  is  performed 
incorrectly. 

Identify  the  malfunction  that  will  occur™ 

The  name  of  a particular  control. 

Identify  how  it  performs  its  function. 

A aariefi  of  component-*  which  perform  par= 
ticul&r  functions. 

Select  the  boat  description  of  how  they 
operate* 
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Qensral  Comment 

In  the  construction  of  the  items  in  the  Terrier  Proficiency 
Teat  Battery,  detailed  care  has  been  taken  to  keep  verbal  facility 
required  by  the  examinee  to  a minimum*  This  has  been  done  by  be- 
ing succinct,  using  job  language,  and  by  using  diagrams  wherever 
applicable*  Fewer  words  are  probably  required  on  the  jobj  however, 
whether  cr  noi,  this  verbal  aspect  significantly  attentuates  the. 
validity  of  teste  uf  this  sort  is  a matter  to  be  investigated* 


The  Content  Covered 


Three  criteria  were  employed  ae  a basis  for  recommendations 
concerning  the  sampling  emphasis  of  the  test  battery  content*  (1) 
The  importance  of  components  in  the  over -all  operation  of  the 
missile  system  as  judged  by  the  project  engineers;  (2)  The  com- 
plexity of  the  job  procedures  concerned  with  a component  or  test 
equipment  as  judged  by  engineers  and  job  analysts)  (3)  The  amount 
and  extent  of  trouble  reported  or  reported  likely  for  certain  com- 
ponents and  equipment  by  engineers  and  operating  personnel* 

Testing  and  Adjustment*  Since  testing  and  adjustment  is 
primarily  concerned  with  the  use  and  interpretation  of  test  equip- 
ment, most  emphasis  in  this  test  is  placed  on  test  and  servicing 
equipment  as  compared  with  missile  components.  Approximately  two- 
thirds  of  the  items  are  primarily  concerned  with  test  and  servicing 
equipment  and  one-third  with  missile  components* 

Troubleshooting*  Trouble-shooting  is  concerned  with  both 
missile  components  and  test  equipment*  On-the-job  performance  is 
primarily  oriented  around  the  location  of  troubles  in  the  missile. 
Two-thirds  of  the  items  in  this  test  are  concerned  with  missile 
cofcponsnts  and  one-third  with  test  and  servicing  equipment. 

Knowledge  of  Operations*  This  test  is  primarily  oriented 
around  the  operation  of  missile  components.  Four-fifths  of  the 
items  are  primarily  concerned  with  missile  components  and  one -fifth 
with  test  and  servicing  equipment. 


The  separation  made  above  between  items  concerned  with 
aissilo  components  and  items  concerned  with  test  and  servicing 
equipment  is  somewhat  arbitrary  and  artificial.  Many  items  are 
not  exclusively  in  one  category  or  the  other,  but  categorisation 
facilitates  the  control  of  the  sampling  of  test  content. 
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A detailed  break-down  of  the  three  tests  listing  the  number 
oi  items  for  each  mis si  la  component  and  separate  pieces  of  test 
and  servicing  equipment  is  as  follows : 


Testing  and 

Trouble- 

Knowledge 

Adjustment 

Shooting 

of  Operations 

Missile  Components 

16 

Receiver 

3 

9 

Outdance  System 
Electrical  System 

5 

10 

(Power  Supply  and 
Switching;  5 

10 

9 

Air  and  Hydraulic 
System 

Qensral  Missile 

2 

5 

5 

i# 

System 

3 

k 

Test  and  Servicing 

Equipment 

Hydraulic  Charging  Unit  u 

3 

3 

Monitoring  Panel 

1? 

9 

7 

Flight  Ready  Indicator  2 

)4 

3 

Guidance  Analyzer 

9 

2 

-2 

50 

57 

73 

Far  the  following  reasons  a number  of  pieces  of  proposed  test 
equipment  to  be  used  with  Terrier  were  not  included  in  the  experi- 
mental test  battery:  (a)  Adequate  technical  materials  were  not 

available  to  the  American  Institute  for  Research,  (b)  The  equip- 
ment was  newly  introduced  in  the  field  or  infrequently  used  by 
operating  personnel,  (c)  Technical  materials  were  received  after 
the  tost  was  prepared  for  administration  at  a particular  time* 
These  pieces  of  equipment  are  the  Beam  Simulator,  the  Beam  Simula- 
tor Test  Unit,  the  Receiver  Test  Unit,  the  Monitoring  Panel  Test 
Unit  and  the  BuOrd  Functional  Test  Equipment  (Hycon). 


Construction  of  the  Tests 


The  construction  of  proficiency  tests  for  personnel  working 
with  complex  equipment  requires  the  coordination  of  engineering 
subject  matter  experts,  field  operating  personnel  and  test  techni- 
cians. Utilizing  these  three  groups  the  steps  involved  in  the 
construction  of  the  Terrier  Proficiency  Te3t  Battery  were  the 
following: 
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1.  Learning  the  System 

The  engineering  staff  studied  the  available  technical 
materials  concerned  with  the  operation  of  the  missile  and  its 
associated  test  equipment  and  learned  the  circuitry  and  operating 
principles  involved*  In  the  course  of  this  study  the  engineering 
staff  developed  a set  of  technical  summary  notes  for  Terrier, 

The  development  of  such  a set  of  notes  is  especially  helpful  for 
a newly  developed  weapon  where  technical  materials  may  exist  in 
discrete  publications  and  require  compilation  and  summarising* 
(These  notes  should  be  especially  helpful  for  curriculum  develop- 
ment} they  are  presented  in  Oulded  Missile  Personnel  Research! 
Report  No*  2>  Part  II,  A.  Compilation  of  Taek  Information  for 
Terrier  Missile  Activities.) 

2.  Learning  the  Tasks  Involved 

Once  the  engineering  staff  became  familiar  with  the  equip- 
ments involved,  they  reviewed  the  specific  procedures  involved  in 
testing  and  adjusting  the  miysilo  with  various  teat  equipments  and 
in  trouble-shooting  and  repairing  the  missile  and  its  associated 
test  equipment*  The  engineering  staff  also  studied  analyses  of 
Terrier  missile  tasks  and  maintenance  data  reports  obtained  from 
operating  installations*  (For  Terrier  task  analyses  and  mainte- 
nance data  reports  see  Guided  Missile  Personnel  Research: Report 
No.  2,  Part  One  and  Part  Two) 

3*  Defining  the  Tests 

On  the  basis  of  an  analyses  of  Terrier  missile  tasks, 
the  test  technicians  specified  the  kinds  of  teats  to  be  developed 
and  described  the  kind  of  test  behavior  to  be  measured  by  particu- 
lar types  of  test  items*  The  tests  and  the  item  types  have  been 
described  r urlier  in  this  chapter*  Each  member  of  the  engineeringg 
staff  was  provided  with  a handbook  describing  good  item  writing 
practices  and  listing  the  item  types  to  be  used  far  each  test;  in 
this  handbook  a sample  ltd was  included  to  illustrate  each  item 
type. 


L*  Defining  Test  Content 


A conference  was  held  between  the  engineering  Item 

m«4  A*«mi  A AaeA  A e4  a na  A a «%1  am  AUa  a anA«***A  i 

mrxWAO  UUJU  VIVO  VVW  V vommav  MNIP  VV  pOAIi  VUD  VV4AVQUU  oili 

matter  coverage  for  each  test*  The  coverage  decided  upon  and  the 
criteria  employed  are  described  earlier  in  this  chapter* 
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Exhibit  A 
ITEM  WRITIN3  FORM 


4.)  Item 

Wv«4  faw  t a 

Name  t b.)  Testi  TA  IS  K 


c.)  Item 

Identification 

Humbert 


d.)  Subject  Matter 


e,  ) Task  or  Principle  Involved 


£ . ) Item  (indicate  correct  answer  by  X}« 
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Exhibit  B 

ITEM  CHECK  LIST  AND  EVALUATION  SHEET 

Test:  TA  TS  K 

Iten  Number 

Reviewer  


1.  The  correct  answer  is:  

2.  Please  check  if  statement  applies*  Use  the  back  of  this  sheet  for  dis- 
cussion of  points  checked.  If  you  check  anything  in  C or  D,  use  either 
the  back  car  the  HIM  WRITING  FORM  to  indicate  the  exact  nature  of  criti- 
cism and  recommended  corrections. 

A.  The  Task  or  Principle  Involved 

( ) a.  Task  or  Principle  Involved  is  not  sufficiently  important. 

( ) b.  Principle  involved  is  important  but  not  required  for  perform- 
ance of  job. 

( ) c.  Item  does  not  test  knowledge  of  stated  principle  cr  performance 
required  by  the  jot, 

( ) d.  Other  (please  specify) 

B.  Level  of  Difficulty 

( ) a.  Item  is  toe  easy. 

( ) b.  Item  is  too  difficult. 

C.  Item  Content 

( ) a.  Additional  information  should  be  supplied  in  stem. 

( ) b.  Problem  needs  clearer  statement, 

( ) c.  More  than  one  answer  could  be  correct. 

( ) d.  Alternatives  are  ambiguous. 

( 5 e.  One  or  more  of  the  incorrect  alternatives  is  very  obviously 
wrong. 

( ) f.  Other  (please  specify). 

D.  Item  Structure 

( ) a.  The  way  the  problem  is  stated  gives  a direct  clue  to  the 
answer, 

( ) b.  Correct  alternative  or  some  other  alternative  steals  out  in 
sane  manner  (length,  wording,  etc.)  from  the  remaining 
answers . 

( ) c.  Other  (please  specify). 

3.  Would  men  Just  out  of  training  school  find  this  item  easier  to  answer 
than  men  who  have  bean  on  the  job  six  months?  Yes  ______  No  ______ 

Why? 

Ur  Recommendations ) check  one: 

( ) a.  Item  la  satisfactory  as  it  now  stands. 

( ) b.  Item  will  be  satisfactory  after  corrections  indicated  teve 
been  made. 

( ) o.  Item  should  be  discarded. 
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$.  Writ 


ths  Item 


On  ths  basis  of  ths  foregoing  preparations.  ths  engineer- 
ing  staff  proceeded  to  writs  test  items.  Particular  individuals 
were  assigned  specific  subject  matter  divisions.  The  first  draft 
of  an  item  was  submitted  on  the  Item  Writing  Form  shown  in  Exhibit 
A.  On  this  form  ths  following  information  was  placed  * 


a.  The  item  writer's  name. 

b.  The  test  for  which  the  item  wan  written.  Testing  and 
Adjustment  (TA).  Troubleshooting  (IS),  or  Knowledge 
of  Operation  (E). 

c.  The  item  identification  number.  An  Item  was  identi- 
fied by  a number  and  by  the  initial  of  the  item 
writer.  If  nr.  Rohrer  was  writing  his  nineteenth 
item,  ite  identification  number  was  R-19. 

d.  The  subject  matter  involved,  i.e.,  the  equipment  or 
circuit  with  which  the  item  was  concerned. 

e.  The  task  or  principle  involved,  i.e.,  the  job  pro- 
cedure or  the  principle  of  operation  tested  by  the 
item. 

f.  The  item  itself  with  the  correct  answer  indicated. 


6.  First  Review 

The  submitted  item  was  reviewed  by  a member  of  the  staff 
who  had  both  engineering  and  test  technician  training.  Comments 
were  made  about  the  technical  correctness  of  the  item  and  about- 
its  structure  in  keeping  with  good  item  writing  practice. 

7.  Item  Conference 

A conference  was  held  with  the  item  writer  to  discuss 
and  review  the  item,  jd  the  basis  of  this  conference,  the  item 
was  either  accepted,  revised,  or  rejected. 

6.  Second  Engineering  Review 

The  accepted  or  revised  item  was  reviewed  by  an  independ- 
ent  engineering  expert.  This  evaluation  was  performed  with  the 
aid  of  the  Item  Check  List  and  Evaluation  Sheet  shown  in  Exhibit 
B.  This  form  is  primarily  self-explanatory.  The  question  under 
3.  in  this  form  was  devised  to  help  detect  items  which  primarily 
tested  school  knowledge  rather  than  job  knowledge. 
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9.  Second  Conference  (if  necessary) 

If  disagreement  existed  between  the  item  writer  and  the 
second  reviewer  on  any  item,  a second  conference  was  held  to  rework 
the  item. 

10.  Item  Editing 


Once  passed  by  the  engineering  staff,  an  item  was  edited 
by  a test  technician  and  prepared  for  inclusion  in  the  test. 


11.  Drafting 


The  diagrams,  waveforms,  pictures,  etc.,  accompanying 
each  item  were  prepared  by  a draftsman  for  inclusion  in  the  test. 

12.  Review  by  Operating  Personnel 

In  this  step  of  test  construction,  the  test  items  were 
assembled  into  subject  matter  sections,  e.g.,  items  on  the  re- 
ceiver, power  supply,  Flight  Ready  Indicator,  etc*,  were  grouped* 
and  each  section  was  discussed  with  operating  personnel  who  were 
judged  by  their  superiors  to  have  experience  with  and  knowledge 
of  the  particular  equipment  in  that  subject  matter  section.  Each 
test  item  was  reviewed  by  a group  interview  procedure.  The  itsa 
was  presented  to  a group  of  Individuals,  and  each  person  marked 
his  answer*  the  correct  answer  was  then  given  and  the  group  mem- 
bers were  asked  to  justify  their  answers  and  to  explain  any 
misunderstandings  they  had  with  the  presentation  of  the  itsa. 

From  this  group  discussion,  any  necessary  suggestions  were  ob- 
tained for  revision  and  improvement  of  the  item.  Personnel  from 
Guided  Missile  Service  Unit  No.  211  at  the  lorfctovn  Naval  Mine 
Depot  and  U.S-  Naval  Guided  Missile  Unit  No.  21  aboard  the  U.S.S. 
Mississippi  cooperated  in  this  review  of  test  items. 

The  review  of  the  test  battery  by  operating  personnel 
accomplished  the  following  purposes i 

a.  The  identification  of  items  that  were  not  job- 
oriented,  e.g.,  items  that  do  not  reflect  current 
job  practices. 

b.  The  identification  of  technically  defective  items 
and  the  determination  of  needed  improvements  and 
revisions. 

c.  The  identification  of  ambiguous  terms,  indeterminate 
items,  and  implausible  distractions  (incorrect 
choices). 

d.  The  identification  of  incorrect  or  defective  aspects 
of  the  circuit  diagrams  and  pictures  accompanying 
the  item. 
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13.  Assembly  into  the  Experimental  Battery 

After  review  by  operating  personnel,  necessary  revisions 
and  improvements  were  incorporated  into  the  test  items  and  the 
items  were  assembled  into  the  test  booklets  of  the  Proficiency 
Examination  far  the  Terrier  fissile  which  is  presented  in  Appendix 
A.  In  order  to  keep  testing  sessions  from  being  too  lengthy,  each 
of  the  three  tests  in  the  battery  was  divided  into  two  parts,  a 
Part  A and  a Part  3. 


The  test  was  first  administered  to  th»  graduating  class  of 
the  U,  S.  Haval  Guided  Missile  School  in  Pomona,  California.  The 
results  of  this  administration  are  being  analyzed  and  will  be 
reported  in  a subsequent  report. 
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CHAPTER  II 

THE  TRCUELE-SBOOTING  FORMBOARD 


As  indicated  in  Chapter  I,  trouble -shooting  behavior  Involves 
the  interpretation  of  symptoms  of  malfunctioning;  on  the  basis  o? 
these  sytgxfcams,  a technician  isolates  a trouble  to  a particular 
component  of  a system.-*-  An  important  aspect  of  trouble-shooting 
behavior  is  that  in  many  situations  the  technician  observes  the 
symptoms  of  malfunctioning  sjv*  performs  ^ series  of  checks  which 
give  Mm  feedback  information.  On  the  basis  of  this  information 
he  can  isolate  the  cause  of  the  trouble.  The  test  construction 
problem  for  behavior  of  this  kind  is  to  build  a test  item  that 
supplies  this  feedback  information.  A type  of  item  called  the  Tab 
Item  has  been  developed  to  do  thin. 2 The  trouble-shooting  form- 
board  discussed  in  this  chapter  is  a modification  of  the  Tab  Item. 
In  addition  to  the  utility  of  the  trouble-shooting  formboard  as  a 
proficiency  examination  device,  it  can  be  of  important  use  as  a 
training  aid.  This  is  especially  desirable  in  the  present  stage 
of  the  Terrier  missile  program  where  it  is  sot  feasible  in  many 
instances  to  obtain  actual  equipment  for  training  purposes. 

The  trouble-shooting  formboard  is  specifically  designed  to 
measure  a technician's  proficiency  in  trouble-shooting  a complex 
system,  it  the  psesent  Instance  the  Terrier  missile  system.  The  trouble- 
shooting formboard  consists  of  several  boards  and  cards  ar  shown 
in  Figures  1 and  2.  These  consist' oft 

1.  A symptom  card.  This  presents  a description  of  a symptom 
of  system  malfunctioning;  see  Figure  3,  and  one  or  more 
of  the  following: 


Trouble-shooting  behavior  for  the  Terrier  missile  is 
described  in  Guided  Missile  Personnel  Research:  Report 

NO.  2 1 Part  T^V  A ^^wryTl  n+.j  .vr  r.f*  Task  T-^n-rnrw»-H  rm  f nr 

Terrier  Missile  Activities. 
o 

~ Glaser,  R.  and  Deunrin,  L,  The  Tab  Item:  A technique 

for  the  measurement  of  proficiency  in  diagnostic  problem 
solving  •basks.  A paper  presented  ai  the  meetings  of  the 
American  Psychological  Associatio  in  Washington,  D.  C., 
September  1-6,  1952  (to  be  published). 
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A trouble -shooting  formboard  problem  (showing  a possible  sol 
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Fig.  2.  A trainee  working  on  a trouble-shooting  forrr.board  problem. 
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Fig,  3,  A symptom  card. 
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Fig.  5 s A unit  board,, 
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2.  A check  board-  This  presents  a series  of  cnccK;  testing; 
and  alignment  procedure e which  a technician  night  employ 
to  determine  the  cause  of  aalfunctioning;  see  Figure  U, 

3.  A unit  board.  This  containB  a list  of  components  or  parts 
in  the  set  which  might..  if  defective,  cause  the  described 
malfunctioning;  see  Figure  5» 

In  the  example  given  in  Figure  L,  it  cao  be  seen  that  a check 
board  contains  the  following  s Or.  the  left  side  of  the  board,  a 
series  of  check  procedures  are  described.  On  the  right  side  oppo- 
site each  of  the  listed  check  procedures  are  rectangles  containing 
the  results  the  examinee  v/ould  obtain  if  he  actually  performed 
this  check  on  an  operating  Terrier  missile.  For  instance,  in  the 
rectangle  opposite  Check  A in  Figure  L,  there  are  several  voltage 
readings  which  he  could  obtain;  and  in  the  rectangle  opposite  Check 
C-l,  there  is  a waveform  he  might  desire  to  see  in  order  to  inter- 
pret the  missile's  performance.  The  information  within  the  rect- 
angles is  not  exposed  to  the  trainee  at  the  time  he  starts  to  work 
on  the  trouble-shooting  formboard;  this  information  is  covered  by 
a block  which  can  be  lifted  off,  see  Figures  I u«a  2. 

In  Figure  5>  which  gives  an  illustration  of  a unit  board,  it 
can  be  seen  that  the  left  side  of  the  unit  board  contains  a list 
of  components  of  the  system.  Either  the  word  "YES"  or  the  word 
"NO"  is  listed  in.  circles  opposite  each  component.  In  this  illus- 
tration the  circle  opposite  ’unit  number  9 has  the  word  "YES"  writ- 
ten in  it;  this  indicates  that  this  is  trie  correct  answer  to  the 
trouble-shooting  problem,  i.e.,  this  is  the  component  that  is  de- 
fective and  is  causing  the  described  symptom  of  malfunctioning- 
The  indications  "YES"  and  "NO"  are  not  exposed  when  a trainee 
starts  to  work  on  the  formiboard;  they  are  covered  by  pegs  planed 
in  each  hole  - When  the  formboard  is  presented  for  uuministration, 
it  is  laid  out  in  front  of  the  examinee  as  in  Figure  2,  which 
shows  an  individual  working  or  u trouble-shooting  problem. 

The  above  description  should  make  evident  that  the  trouble- 
shooting formboard  attempts  to  reproduce  a realistic  trouble- 
shooting situation  in  a training  situation  without  the  use  of  ac- 
tual ecruipsaent,  which  is,  in  many  instances,  difficult  and  expen- 
sive to  obtain  and  expensive  and  vime -consuming  to  maintain.  The 
behavior  required  of  an  individual  working  on  a formboard  appears 
to  approximate  closely  some  essential  aspects  of  the  behavior  re- 
quired in  a trouble-shooting  Job.  In  the  Job  situation,  a tech- 
nician generally  receives  a report  of  malfunctioning  through  a 
written  or-  verbal  report,  or  by  indications  on  test  or  alignment 
equipment  with  which  he  is  working.  Re  then  proceeds  to  check 
through  the  system  in  an  attempt  to  discover  and  isolate  the  c&use 
of  the  trouble.  In  the  trouble-shooting  formboard,  the  technician 
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is  presented  with  a description  of  malfunctioning;  and  then,, 
rather  than  actually  performing  various  check  procedures,  he 
lifts  off  the  blocks  of  those  checks  he  would  make  if  he  were 
trouble-shooting  a real  system.  Whenever  he  lifts  a block,  i.e., 
makes  a check,  he  receives  the  information  he  would  obtain  if  he 
actually  performed  that  check  on  a functioning  system.  When  the 
trainee-  pulls  a peg  on  the  unit  board,  lie  It)  ’nhfetbsr  or 

not  a particular  cosgonant  is  defective.  The  word  "YES"  informs 
him  that  the  component  is  defective  and  that  he  has  solved  the 
problem  by  locating  the  defective  component,  lfce  word  "NO"  in- 
dicates that  the  particular  component  examined  is  not  defective 
and  trouble-shooting  must  continue,  The  words  "YES"  and  TO 
duplicate  the  on-the-job  procedure  of  checking  a coeponent  which 
the  technician  has  reason  to  believe  might  be  faulty.  For  ex- 
ample,  a tube  might  be  tested  and  found  to  be  short-circuited 
thus  indicating  "YES,  this  is  the  cause  of  the  trouble."  On  the 
other  hand,  the  tube  might  test  out  normally,  thus  indicating 
that  "NC,  this  is  not  the  cause  of  the  trouble." 

In  working  on  the  trouble-shooting  forniboard,  the  trainee  is 
instructed  to  locate  the  defective  unit  by  making  as  few  checks 
as  possible.  He  is  free  co  select  any  single  check,  or  combina- 
tion of  checks,  he  thinks  necessary  to  obtain  the  information 
which  he  needs  to  isolate  the  trouble  to  a specific  unit.  When 
he  reaches  this  point,  be  Fulls  the  peg  beside  the  selected  unh- 
and is  informed  as  to  the  correctness  of  his  diagnosis.  If  he 
finds  a "YES",  he  has  located  the  trouble  has  solved  that 
particular  problem.  If  he  finds  a "NO",  he  is  free  to  go  back 
and  obtain  additional  information  by  performing  additional  check 
procedures  before  n»king  another  diagnosis.  The  trainee  is  spe- 
cifically instructed  that  he  will  be  penalized  for  m^k-t  ng  erro- 
neous diagnoses,  i.e.,  pulling  "NO"  unit  peg3j  this  is  done  to 
minimize  the  location  of  the  "YES"  unit  by  guessing.  Detailed 
instructions  for  administering  the  trouble-shooting  forniboard 
are  presented  in  Appendix  C. 


Construction  of  the  Trouble-Shooting  Forniboard 

As  indicated,  the  material  presented  in  a trouble-shooting 
forniboard  consists  of  the  following: 

A.  The  synptom  of  malfunctioning 

B.  The  check  procedures 

C.  The  results  of  the  check  procedures  (under  blocks) 

D.  Ihe  possibly  defective  components 
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Specific  considerate ons  concerning  the  construction  of  each  of 
these  will  new  be  discussed. 

A.  The  symptom  of  malfunctioning.  The  description  of  the 
synptem  must  be  clearly  and  carefully  presented.  Ambiguity  in  the 
description  can  force  the  trainee  to  perform  unrealistic  checks 
for  the  purpose  of  clarifying  the  exact  nature  of  the  malfunction- 
ing rather  than  for  the  purpose  of  isolating  the  defect.  If  this 
occurs,  a measure  of  the  trainee's  ability  to  understand  what  the 
constructor  of  the  formboard  problem  meant  is  obtained  in  addi- 
tion to  his  ability  to  trouble-shoot.  A symptom  is  best  presented 
in  terms  of  the  specific  test  equipment  and  operating  indications 
that  are  part  of  the  Job.  Wherever  applicable,  a diagram,  picture, 
or  check  sheet  should  be  used  to  present  the  symptom. 

B.  The  check  procedures.  The  checks  included  in  each  form- 
board  item  must  present  a- wide  variety  of  trouble-shooting  pro- 
cedures--from  those  wka  are  very  relevant  to  the  detection  of 
the  cause  of  the  trouble  involved,  to  those  which  are  quite  irrel- 
evant* This  serves  the  double  purpose  of  (l)  presenting  the 
trainee  with  a situation  that  retains  some  of  the  unstructured 
aspects  of  the  actual  Job  situation,  and  (2)  permitting  the  vide 
range  of  scores  necessary  for  discriminating  among  several  levels 
of  proficiency.  These  considerations  have  led  to  the  inclusion 
of  checks  of  the  following  types,  classified  according  to  the  in- 
formation which  they  provide: 

1.  Relevant.  Information  that  is  pertinent  or  essential  to 
tfve  detection  of  the  faulty  component.  (A  very  profi- 
cient mechanic  would  utilize  these  procedures. ) 

2.  Additional.  Extra  information  that  a less  proficient 
mechanic  might  need  for  solution  of  the  problem,  but 
that  a highly  proficient  cue  could  infer  from  either 

the  description  cf  malfunctioning  or  the  relevant  checks. 

3*  Redundant.  Information  which  duplicates  the  informa- 
tion given  by  a previously  performed  check. 

h.  Inadequate.  Information  which  is  contained  as  part  of 
the  information  *ri  ven  by  other  more  cornprehenslve  checks 
of  the  relevant  type. 

p.  Irrelevant.  Information  which  is  completely  unrelated 
to  the  solution  of  /ulem.  (On  ly  mechanics  with  a 

very  low  degree  of  proficiency  would  utilize  tnese  pro- 
cedures. } 

Ca  The  results  of  the  check  procedures.  As  in  the  descrip- 
tion of  malfunctioning,  the  results  given  under  the  block  for  each 
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check  in  the  item  must  he  unambiguous  and  technically  accurate. 

To  help  insure  freedom  from  ambiguity,  a picture  or  diagram  is 
given  wherever  possible.  The  use  of  a picture  or  a diagram  also 
serves  to  lessen  the  importance  of  the  trainee's  ability  to  visual- 
ize verbally-presented  material.  An  obtained  result  may  be  shown 
in  comparison  with  the  normal  result  or  it  my  be  stated  that  the 
obtained  result  is  in  or  out  of  to.leranct.  This  is  desirable  for 
results  which  a trainee  would  not  be  expected  to  interpret  offhand 
on  the  job  and  would  require  that  he  refer  to  a manual  or  check-list. 

D.  The  possibly  defective  components.  Considerations  similar 
to  those  mentioned  for  toe  construction  of  the  cheek  procedures 
are  necessary  in  selecting  the  units  to  be  included.  The  units 
from  which  the  trainee  is  to  select  in  malting  his  diagnosis  are 
of  the  following  kinds: 

1.  Correct.  The  component,  that  is  causing  the  described 
trouble. 

2.  Plausible.  Components  which  could  cause  the  described 
trouble,  but  which  can  be  eliminated  by  performing  cer- 
tain combinations  of  checks  given  in  the  item.  For  each 
plausible  unit  there  exist  one  or  more  check  procedures 
which,  if  performed  and  if  properly  interpreted,  will 
permit  the  trainee  to  eliminate  "chat  component  from  con- 
sideration. 

3*  Irrelevant.  Components  which  could  not  possibly  cause 
the  described  trouble. 

In  actual  physical  construction,  the  trouble-shooting  form- 
board  can  consist  of  a hardboard  or  sheet-metal  frame.  The  frame 
consists  of  a front  panel  with  rectangles  (or  holes)  cut-  out  as 
shown  in  Figure  6 and  7;  the  hack  panel  is  a solid  panel  with  no 
cut-out  apertures.  The  front  and  bad':  panels  are  riveted  together 
with  enough  space  between  the  two  tc  slide  a card  made  of  stiff 
cardboard.  Figure  6 shows  an  item  card  being  inserted  into  a 
check  board.  The  blocks  for  check  boards  r .e  made  of  wood  and  the 
pegs  for  unit  boards  can  be  cut  from  wood  dowels.  The  blocks  and 
pegs  should  be  painted  a dark  color  so  that  markings  and  finger- 
prints will  not  distinguish  the  most  popularly  used  checks 
units.  Detailed  specifications  for  the  construction  of  check  pnfl 
unit  formboards  are  given  in  Figure  7* 

A single  trouble-shooting  forinboard  item  consists  of  a eyeptran 
card,  one  or  more  check  boards,  and  one  or  more  unit  boards  depend- 
ing upon  the  number  of  check  procedures  and  units  involved  in  a 
particular  problem.  Each  problem  is  set  up  on  cards,  a synptom 
card  end  check  and  unit  cards;  the  check  and  unit  cards  are  in- 
serted into  chock  and  unit  boards.  For  every  new  problem  or  fora- 
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6<>  Inserting  an  item  card  in  a formboard  .frame 0 
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board  item  presented  to  a trainee,  new  cards  are  used  and  the  neces- 
sary number  of  boards  are  euplcyed.  It  is  advisable  to  keep  the 
number  of  check  beards  to  about  four  or  five  or  less  and  the  number 
of  unit  boards  to  one  or  two  or  else  the  situation  becomes  un- 
wieldy and  too  much  material  is  presented  to  a trainee  at  one  time. 

Each  set  of  cards  that  make  up  one  trouble-shooting  problem 
can  be  considered  as  one  test  item  and  the  thirteen  steps  described 
in  Chapter  1 for  the  development  of  Terrier  multiple -choice  items 
were  employed  for  the  development  of  the  formboard  items.  Experi- 
mental trouble-shooting  formboard  items  developed  for  the  Terrier 
missile  are  presented  in  Appendix  B. 


With  respect  to  the  content  coverage  of  these  items,  most  of  the 
formboard  items  involve  more  than  one  of  the  sub -systems  of  the 
missile  or  associated  test  equipment.  The  specific  malfunctioning 
equipment  and  defective  unit  with  which  each  item  is  concerned  is 
listed  below: 


Item  No. 


Malfunction! ng  Equipment 
and  the  Defective  ImlF 


1 receiver  package,  reference  discriminator 

2 receiver  package,  automatic  gain  control 

3 receiver  package,  crystal  mixer 

4 guidance  package 

5 guidance  package,  "B'!  channel  reference  driver 

6 guidance  package,  first  limiter 

7 power  supply  package 

8 power  supply  package,  choke 

9 electrical  system,  power  change  - over  relay 

10  hydraulic  system,  autopak 

11  hydraulic  system,  alternator-hydraulic  motor 

xd  hydraulic  system,  hydraulic  valve-rolleron 

actuator 

13  hydraulic  system,  "A"  channel  hydraulic  valve 

14  monitoring  panel,  resistor  in  dc  amplifier 

15  monitoring  pane a,  transformer  in  electronic 

chassis 


Scoring  the  Trouble-Shooting  Formboard  It^mw 


Severed  methods  for  scoring  performance  on  a trouble-shooting 
formboard  problem  are  under  consideration.  The  empirical  data  ob- 
tained from  the  application  of  the  various  scoring  procedures  uni  t 
be  compared  and  evaluated. 
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A proficient  trouble-shooter  may  bj  assumed  to  be  one  who  will 
detect  a source  of  nmlfurctioning  most  efficiently,  with  a minimum 
of  activity.  On  this  assumption,  when  working  on  the  troubles  host- 
ing f ormboard,  he  will  select  a minimum  number  of  relevant  check 
procedures  end  proceed  to  identify  the  defective  component  with  a 
minimum  number  of  pulled  pegs.  A poor  trouble-shooter,  on  the  other 
hand,  may  be  considered  as  one  who  will  need  more  information,  will 
perform  non-essential  procedures,  and  will  make  more  erroneous 
diagnoses.  Accordingly,  the  simplest  method  of  scoring  a f ormboard 
trouble-shooting  problem  is  a straight  frequency  count  of  the  com- 
bined numb*?  of  the  blocks  and  pegs  pulled.  In  this  method,  the 
greater-  the  score  the  poorer  the  performance  of  the  trainee. 

A more  complex  method  involves  a weighted  error  count  wherein 
the  procedures  and  units  which  can  be  selected  are  weighted  in  in- 
verse relation  to  their  relevance  in  isolating  the  defective  unit. 
The  weight a are  based  upon  the  kinds  of  checks  and  components  which 
the  examinee  selects.  (See  8 and  D under  Construction  of  the  Trou- 
ble-Shooting Formboard, ) With  respect  to  the  check  procedures, 
relevant  checks  ere  weighted  minimally  and  irrelevant  ones  maxi- 
mally. Additional  checks  receive  a weighting  intermediate  between 
these  two.  Redundant  and  inadequate  checks  by  definition  are 
meaningful  only  in  terms  of  related  procedures  and  thus  are  weighted 
only  as  thsy  appear  in  pre-determined  combinations.  For  example, 
if  a check  has  been  performed  which  gives  certain  information  and 
then  a subsequent  check  is  performed  which  give-  identical  infor- 
mation, the  latter  check  is  redundant  and  receives  extra  weight. 

With  respect  to  the  units,  the  relevant  unit  receives  a jero 
weighting  and  irrelevant  units  receive  maximum  weighting.  Plausi- 
ble units  are  weighted  so  as  to  penalise  guessing;  if  a "NO"  unit 
peg  is  gulled  and  the  check  procedures  which  relate  to  it  have 
not  been  pulled,  the  unit  is  weighted  maximally  on  th9  assumption 
^at  the  trainee  is  guessing. 

A somewhat  different  scoring  technique  is  based  upon  the  fact 
that  a very  proficient  trouble-shooter  will  isolate  a defect  in 
the  shortest  possible  time.  Weighting  of  checks  and  units  can  be 
based  on  the  time  required  to  perform  check  procedures  and  to  get 
at  and  examine  possible  defective  units.  With  this  scoring  method, 
the  trainee  is  instructed  that  his  performance  or?  the  trouble-shoot- 
ing forraboerd  will  be  scored  on  the  basis  of  the  tias  required  to 
perform  the  identical  operations  with  the  actual  equipment.  Hs  re- 
ceives least  weight  (less  time  penalty)  for  those  checks  and  units 
he  selects  which  require  the  least  amount  of  time  to  perform  on  the 
job.  A man  who  performs  a trouble-shooting  job  more  efficiently 
in  terms  of  the  shortest  amount  of  tins  receives  a low  time  score. 
This  scoring  procedure  introduces  a desirable  realistic  bias,  to 
the  job  a technician  will  often  perform  quick  and  simple  checks 
before  undertaking  an  involved  and  ccvylox  procedure,  and  he  may 
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alBO  investigate  easily  accessible  uuits  first  before  Investigating 
units  which  are  more  difficult  to  get  at.  If  these  checks  give  the 
technician  the  information  he  needs,  he  may  save  the  time  required 
to  perform  the  more  lengthy  checks.  If  the  quick  and  single  checks 
do  net  provide  him  with  the  necessary  information,  he  has  lost 
only  a short  amount  of  time  arid  can  precede  with  the  more  time-con- 
suming checks.  This  scoring  method  takes  into  account  a man's 
ability  to  consider  the  "time  efficiency"  in  his  trouble-shooting 
behavior.  The  feasibility  of  the  procedure  is  being  studied. 

Trouble-shooting  and  the  trouble-shooting  formbosrd  involve 
the  presentation  of  an  information  series:  and  the  possibility  of 
the  application  of  the  concepts  of  a recently  developed  mathematical 
theory  of  information  has  been  studied  (Shannon,  C.  £*  and  Weaver 
W.  The  Mathematical  theory  of  conaunlcatlon.  Hie  University  of 
Illinois  ^ress:  Orbana,  Illinois,  1^+9  )•  ^procedure  has  been  de- 
veloped for  scoring  a trouble-shooting  problem  based  on  this  theory - 
Discussion  of  this  is  presented  a*  a separate  technical  note  in 
Appendix  D. 


General  Carcnents 


An  injportant  limitation  of  the  trouble-shooting  formhoard  is 
that  it  does  not  measure  the  manipulative  skills  involved  in  the 
performance  c-f  a check  procedure.  In  addition,  it  makes  demands 
upon  the  trainee's  verbal  ability  and  upon  hi  a ability  to  visual- 
ize the  functioning  of  & piece  of  complex  equipment  when  it  is  not 
actually  before  ham.  A particular  3et  of  behaviors  ie  required 
for  a mechanic  to  actually  perform  a series  of  control  settings 
and  then  measure  a specific  voltage  output,  while  a somewhat  dif- 
ferent, but  related,  set  of  behaviors  is  necessary  for  him  to 
read  that  certain  control  settings  have  been  made  and  then  select 
a particular  procedure.  This  discrepancy  between  test  behavior 
and  job  behavior  is  not  unique  to  formhoard  items,  and  validation 
data  usually  inform  us  of  the  seriousness  of  this  discrepancy. 

Since  a single  formhoard  item  yields  a range  of  scores  rather 
than  the  usual  pass-fall  indication  for  a test  item,  seme  problems 
arise  concerning  the  application  of  conventional  item  statistics. 
It  is  possible  to  employ  the  techniques  based  on  dichotomous  re- 
sponses by  establishing  an  arbitrary  pass-fall  cutting  point  in 
the  range  of  item  scores,  but  such  dichotomizing  of  data  may  sac- 
rifice useful  information.  An  alternative  procedure  for  computing 
an  index  of  item  difficulty  is  to  use  the  average  number  of  pulls 
required  by  a group  of  trainees  for  each  item.  Subtracting  this 
average  score  from  the  minimum  number  of  pulls  necessary  to  solve 
an  item  yields  the  average  amount  of  the  deviation  from  a perfect 
score.  Ms  or  some  function  of  it  can  be  taken  as  an  index  of 


••  23  «• 

•!AU  SECURITY  INFORMATION 


v . 


oECURlTY  INFORMATION 


item  difficulty.  The  minimum  number  of  pulls  is  determined  on  the 
basis  of  export  opinion  and  analysla  of  item  responses.  For  the 
computation  of  item  validity,  there  appear  to  be  no  special  prob- 
lems. The  entire  range  of  item  scores  nay  be  used  in  correlations 
with  criterion  scores  if  desired. 

In  conclusion,  it  should  be  emphasised  that  the  trouble-shoot- 
ing foraboard  can  be  utilized  both  as  proficiency  measurement  pro- 
cedure and  as  a training  aid.  During  periods  whan  equipment  is 
scarce  for  training  purposes,  the  use  of  the  foraboard  may  offer 
an  inexpensive  and  practical  procedure  for  training  in  trouble- 
shooting techniques. 
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